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Abstract 


The presence of robots in agriculture has grown significantly in recent years, overcoming 
some of the challenges and complications of this field. This chapter aims to collect a com- 
plete and recent state of the art about the application of robots in agriculture. The work 
addresses this topic from two perspectives. On the one hand, it involves the disciplines that 
lead the automation of agriculture, such as precision agriculture and greenhouse farming, 
and collects the proposals for automatizing tasks like planting and harvesting, environ- 
mental monitoring and crop inspection and treatment. On the other hand, it compiles and 
analyses the robots that are proposed to accomplish these tasks: e.g. manipulators, ground 
vehicles and aerial robots. Additionally, the chapter reports with more detail some practi- 
cal experiences about the application of robot teams to crop inspection and treatment in 
outdoor agriculture, as well as to environmental monitoring in greenhouse farming. 


Keywords: robotics, agriculture, greenhouse, UGV, UAV, multi-robot 


1. Introduction 


Agriculture can be a field as favourable as industry for the application of automation. The 
challenges for robots in agriculture are diverse. On the one hand, agricultural environments, 
in contrast to industrial facilities, are not structured and controlled. On the other hand, indus- 
trial processes can be designed by modules to apply specific robots to specific works, whereas 
the complex tasks of agriculture sometimes cannot be split into simple actions. For the above 
reasons, agricultural applications require more versatile and robust robots. 
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In the last years, multiple groups around the world have applied different automation 
solutions (e.g. sensor networks, manipulators, ground vehicles and aerial robots) to diverse 
agricultural tasks (e.g. planting and harvesting, environmental monitoring, supply of water 
and nutrients, and detection and treatment of plagues and diseases). This chapter aims to 
collect the state of the art about robotics applied to agriculture, as well as to describe more 
exhaustively some of our practical experiences. 


Section 2 addresses the agricultural tasks where the robots can be applied, remarking preci- 
sion agriculture and greenhouse farming, but covering other works such as automatic plant- 
ing and harvesting. Section 3 describes the robots applied in agricultural tasks in the state of 
the art, covering both multi-robot systems, ground and aerial robots. Section 4 summarizes 
our main experiences in robotics applied to agriculture, which are related to precision agricul- 
ture in open fields and environmental monitoring of greenhouses. Finally, Section 5 summa- 
rizes the main conclusions acquired from the review of the state of the art and our experience 
in the research projects. 


2. Automation in agriculture 


This section reports the state of the art about automation in agriculture. For this purpose, it is 
organized as follows: Section 2.1 is focused on precision agriculture, Section 2.2 addresses the 
application of new technologies to greenhouse farming, and Section 2.3 analyzes the propos- 
als for automatic planting and harvesting. 


2.1. Precision agriculture 


Precision agriculture, also known as precision farming, is a concept of farm management 
based on the application of different technologies, in order to manage the spatial and tem- 
poral variability associated with all aspects of agricultural production. Its main goal is the 
improvement of both crop performance and environmental quality. 


Several authors have confirmed the economic and environmental benefits that are 
achieved when precision agriculture methodologies are applied. Nonetheless, academic 
surveys and professional reports show that the rate of adoption of these technologies is 
still low [1]. 


Moreover, instead of using precision agriculture as complete concept, most of the deploy- 
ments reported use these techniques to solve specific needs or to fill important gaps in the 
knowledge of farmers [2]. Additionally, even though agronomists are playing the leading 
role in PA development, engineers have worked diligently to provide technologies needed 
to implement PA practices. Engineering innovations for PA involve development of sensors, 
controls, and remote-sensing technologies. 


Autonomous mobile robots can be used in a variety of field operations. They can be applied 
to facilitate capturing and processing high quantities of data, and they can provide the capa- 
bilities required to operate not only at individual plant level but also at complete field level. 
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Blackmore and Griepentrog [3] study the autonomous platforms that may be available in the 
future, which would be used for cultivation and seeding, weeding, scouting, application of 
fertilizers, irrigation and harvesting. 


The most widely used robotic technology in precision agriculture is vehicle guidance and auto- 
steer systems. The reason is that the economic benefits are easily achievable without requiring 
the integration of additional components or decision support systems [2]. However, other 
technologies, especially those related to remote sensing, development of sensors and controls, 
are also used by teams combining agronomists and engineers. Table 1 summarizes some of 
the developments, which use mobile robotics and remote sensing for precision agriculture. 


2.2. Greenhouse farming 


Greenhouse farming is often a suitable field for applying the technologies of automation, 
computing and robotics. Some examples of technologies implemented in productive green- 
houses are the control of temperature and humidity, the soil preparation and the supply of 
water and nutrients [10]. The robots can perform some tasks that humans cannot do due to 
the harsh conditions of greenhouses, such as environmental monitoring and control, crop 
monitoring, supply and treatment, and pest and disease detection. 


The environmental monitoring of greenhouses is interesting not only to control the growth 
of crops but also to determine the traceability of products. Nowadays, most of the systems 
used for environmental monitoring of greenhouses are based on wireless sensor networks 
(WSNs) [11-13]. Nevertheless, the robots are starting to be applied as mobile platforms for 
sensors [14-16]. 


Publication Operation Technique 


[4] Weeding Automatic computer vision method for detecting weeds in cereal 
crops, and differential spraying to control the weeds. 


[5] Field mapping Creation of 3D terrain maps by combining the information 
captured with a stereo camera, a location sensor, and an inertial 
measurement unit, all installed on a mobile equipment platform. 


[6] Field mapping and coverage An unmanned car-like mobile robot uses a SLAM algorithm to 
navigate in the agricultural environment while creating a map of 
such environment. 


[7] Multi-purpose Design and construction of a multi-purpose mobile ground 
platform for PA tasks. 
[8] Coverage path planning Harmony Search (HS) algorithm for finding complex coverage 


trajectories with a fleet of aerial robots. 


[9] Weed and pest control A fleet of heterogeneous ground and aerial robots is developed 
and equipped with innovative sensors, enhanced end-effectors 
and improved decision control algorithms to cover a large variety 
of agricultural situations. 


Table 1. Precision agriculture: main operations and techniques. 
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Greenhouses can be considered complex multiple-input multiple-output systems [17]. The 
literature collects multiple proposals for modelling and controlling the conditions of green- 
houses [18]. Some of them obtain the models of greenhouses applying analytical equations 
(e.g. mass and energy balances) [19], whereas the rest identifying process models (e.g. neural 
networks or fuzzy sets) [20]. A review of these models determined the input, output, and 
disturbance variables described below. 


The input variables allow to actuate the greenhouses and change the environmental condi- 
tions. The most relevant variables considered by literature are the ventilation [21], heating 
[22], fogging [23], shading and CO, injection [24] systems. The ventilation systems control the 
exchange of air between greenhouse and environment, which has an impact on the air tem- 
perature, humidity and composition. The heating systems are used to compensate the heat 
losses and keep the temperatures in the adequate range. The fogging systems spray water into 
the air to increase the humidity and reduce the temperature. The shading systems control the 
irradiation of the covers to avoid the overheating of the greenhouse. Finally, the CO, injection 
systems are used to promote the photosynthesis of the plants. 


The output variables define the state of greenhouse can be measured by the appropriate sensors 
and are the target of climate control. The most relevant variables collected by literature are the 
air temperature, air humidity, solar radiation and CO, concentration. In addition, there are some 
variables that have influence on the state of greenhouse and should be measured and controlled. 
These disturbances are the external temperature, external humidity, wind speed, wind direc- 
tion, external CO, concentration, cover temperature, crop temperature and ground temperature. 


Table 2 collects the relevant variables for the environmental monitoring of greenhouses [25], 
as well as the appropriate sensors to measure them and their possible application in robots. 


Another task of greenhouse farming where the robots can perform an important role is the 
crop inspection and treatment. The detection of weeds [26], pests [27] and diseases [28] is pos- 
sible through direct and indirect methods. The direct methods are based on acquiring RGB 
[29] and 3D [30] images and applying computer vision techniques. The indirect ones require 
to take samples in the greenhouse and to analyze them in the laboratory. The ground robots 
can be used to apply treatments and fertilizers to the crops [31-33], in order to improve the 
precision and rationalize the products. 


Planting and harvesting are seasonal tasks that require a considerable amount of work. The 
literature contains some proposals to automatize these tasks [34-36]. These proposals con- 
sider different types of robots (ground mobile and rail robots), sensors (mainly, RGB and 3D 
cameras and laser scanners) and effectors (manipulators and graspers). 


2.3. Seeding and harvesting 


Regarding the point of view of service robotics, Onwude et al. [37] attempt to evaluate the 
application of agricultural mechanization and its present technologies and limitations for 
the large-scale purposes. The study shows an increasing level of technological advancement 
in field and crop mapping, soil sampling, mechanical seeders and harvesters in agricultural 
robots. Sharing the same approach, Kester et al. [38] show the future trends and the likely 
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Variable Sensor Application 
Radiation (absorbed) Net radiometer AR, GR, FS 
Radiation (solar) Pyranometer AR, GR, FS 
Air temperature Resistance temperature device AR, GR, FS 
Thermocouple AR, GR, FS 
Thermistor AR, GR, FS 
Surface temperature Infrared AR, GR, FS 
Thermocouple GR, FS 
Substrate temperature Resistance temperature device GR, FS 
Thermocouple GR, FS 
Thermistor GR, FS 
Air humidity Capacitance hygrometer AR, GR, FS 
Condensation hygrometer AR, GR, FS 
Psychrometer AR, GR, FS 
Ground humidity Electrical conductivity meter GR, FS 
Carbon dioxide in air Non-dispersive infrared sensor AR, GR, FS 
pH pH probe GR, FS 


AR, air robot; GR, ground robot; FS, fixed sensor. 


Table 2. Environmental monitoring of greenhouses: variables, sensors and robots based on Ref. [16]. 


adoption of automated farming machinery. The results of this study point out a growing 
interest in autonomous and semi-autonomous systems for reducing the highest workload 
operations: tillage, seeding and harvesting. 


In order to support the growing development of seeding and harvesting robots, new strategies are 
proposed for driving autonomous mobile robot in the associated scenarios. Ko et al. [33] makes a 
small review about common techniques for robot navigation in greenhouses and proposes a meth- 
odology based on machine learning. Due to the importance of the manipulation tasks in seeding, 
transplanting and harvesting processes, the manipulator motion strategy is also discussed by sev- 
eral authors. However, they usually do not specify path planning algorithms; the most common 
approach is the direct displacement towards the end-effector desired position by position-based 
control [39] and visual feedback control [40]. The task planning strategies are only studied by a few 
researchers. Commonly, the harvesting task is limited to pick one fruit, while the planning required 
for picking the rest is avoided. Nevertheless, the problem can be studied from two perspectives: 
the coverage path planning [41] for picking all the fruits in a scene or the time minimization [42] 
for moving from a fruit to another. The obstacle detection and avoidance are studied by an equally 
low number of authors. A great complexity is added to the solution due to the obstacle recognition 
in addition to the path planning algorithms. A few approaches are based in the obstacle detection 
with collision sensors in the end effector [43], the obstacle recognition by Light Imaging, Detection 
and Ranging (LIDAR) [44] and vision techniques [45]. 
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A review about the use of technologies for automated activities in greenhouses is presented in 
Ref. [46], showing an increasing implementation of wireless systems for environmental mea- 
surements. Additionally, the study shows that a considerable number of research works are 
aimed to develop robotic systems for fruit picking and extraction. Furthermore, the research 
community has put much effort on developing techniques for robust fruit recognition; more- 
over, there is a high necessity for improving the picking capabilities of transplanting and har- 
vesting robots in order to move towards a commercial application. A review on vision control 
techniques and their potential applications in fruit or vegetable harvesting robots is presented 
in Ref. [47]. The fruit identification and localization are the most common problem studied by 
the authors. Like the fruit ripeness identification [48, 49], a great number of approaches are 
based on RGB cameras [50], as well as colour and shape recognition [51]. The multi-spectral 
lightning is less studied despite more information can be acquired with this kind of technol- 
ogy [52]. The next level of complexity involves the implementation of stereo vision systems 
and LIDAR for calculating the fruit position in a 3D space [53]. 


This objective can only be accomplished by the diversification and specialization of the robotic 
systems. With the aim of getting better results in harvesting tasks, newer and more precise 
sensors are needed. In Ref. [36], the authors review the modern sensor systems used in semi 
or fully automated robotic harvesting. Their research shows how the integration of several 
kinds of technologies and sensor fusion can improve the precision in fruit recognition and 
localization activities. 


Some interesting publications to be considered for a further review of seeding, transplanting 
and harvesting robotics can be found in Refs. [34, 54]. 


3. Agricultural robots 


This section reviews the different types of robots and payloads that are applied to agricultural 
tasks. For this purpose, it is organized as follows: Section 3.1 analyzes the aerial robots, Section 
3.2 the ground robots, Section 3.3 some special robots that are not conventional ground or 
aerial vehicles and Section 3.4 the multi-robot systems. 


3.1. Aerial robots 


The Association for Unmanned Vehicle Systems International (AUVSI) published an econom- 
ical report in 2013 [55], which emphasized the future impact of the civil use of the unmanned 
aerial vehicles (UAVs) on the USA economy. This document highlights two markets over 
multiple areas: public safety and precision agriculture. It concludes that by far and above, this 
last market will be the largest in the next decade in terms of economy and jobs. 


In the last decades, collecting data from agricultural holdings has been carried out mainly by 
manned aerial vehicles, satellites or directly specialized experts at floor level [56]. The avail- 
ability of these methods has some limitations, such as the presence of clouds, the long data 
delivery times, the need for special permissions and the prizes of some products. In contrast, 
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the UAVs can be deployed efficiently, can carry multiple types of sensors, do not require very 
restrictive permissions and are becoming a cost-effective alternative. These advantages have 
been reinforced with the rise of the vertical take-off and landing (VTOL) vehicles and more 
specifically the quadcopters (a good example can be seen in Ref. [57]). 


One of the first applications of UAVs in precision agriculture was their use to measure the 
water stress in agricultural holdings. Nowadays, the UAVs are equipped with thermal and 
hyperspectral cameras, as well as fluorescence sensors [58]. An interesting experiment is 
reported in Ref. [59], where the authors produced a controlled deficit of irrigation to gener- 
ate a gradient of water stress in a citrus orchard. They compared the data obtained by the 
micro-thermal and hyperspectral cameras boarded on a fixed-wing unmanned aircraft with 
the measurements on the leaves, validating the aerial methods to measure the water stress. 
Similar works can be found in Refs. [60-62], which show the feasibility and benefits of the 
aerial thermal imagery to improve the irrigation. 


Another use of aerial vehicles in precision agriculture is the monitoring of crops to predict 
yields, properly calculate the amount of fungicides and fertilizers and detect pathogen. In 
Ref. [63], a RGB camera was used to estimate the biomass of a barley exploitation under two 
different nitrogen treatments. The results were cross-validated with five different crop surface 
models based on the height of the plant. Additionally, in Ref. [64], techniques of clustering 
were applied to estimate the biomass of a wheat exploitation through the RGB images. The 
method was enhanced by measuring the height of the plant directly on ground. These studies 
demonstrate the feasibility, applicability and precision of this tool. 


Another interesting parameter for monitoring is the leaf area index (LAI), defined as the ratio 
of leaf surface to unit ground area. In Ref. [65], the authors estimate the LAI of maize, potato 
and sunflower fields by using a hyperspectral camera and inverting the solar model of the 
canopies. In [66], they used a visual 3D modelling technique to estimate the LAI in a vine- 
yard, obtaining around 57% less precision. Although this method seems to be nowadays less 
accurate, it is quick, inexpensive and practical compared to other methods. In Ref. [67], they 
use four cameras and overlap the corresponding images to measure the LAI over wheat and 
rapeseed crops. They achieve a very good correlation in comparison to the measurements at 
floor level, despite that they highlight the dependency of the method on the light conditions. 


Additionally, the UAVs are a practical tool for the prevention of diseases in agricultural hold- 
ings. The work reported in Ref. [68] uses a UAV and a manned aircraft equipped with multi- 
spectral cameras. It applies four classification algorithms to detect the prints of the diseases 
and compare the results with the ground truth. Similarly, the work described in Ref. [69] 
proposes a new airborne visual sensor to detect diseases on the leaves on citrus trees, and the 
experiments show an accuracy of 93%. These methods have been used not only for citrus trees 
but also for olives [70], avocado trees [71], potatoes [72] and grapevines [73], among others. 


Although this use is not extended around the world due to the restrictions in many countries, 
there have been some trials and investigations with UAVs spraying locally fertilizers and 
pesticides [74]. For instance, in Ref. [75], a net of pesticide sensors are deployed over a virtual 
field, whereas a UAV sprays the chemicals where they are needed. The sensors provide a 
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feedback to the UAV, which adjusts the optimal route and minimizes the waste of pesticides. 
A more practical and exhaustive study was carried out in Ref. [76], where different spraying 
parameters are tested in a field with rice crops, such as operation height and velocity, and the 
deposited volume at different heights of the plants. 


As we have seen, the UAVs equipped with hyperspectral and thermal cameras are an afford- 
able and practical system to improve the performance of agricultural holdings, reducing costs 
and increasing productivity. With the development of new sensors and the establishment of 
new legal frameworks, their expansion will continue in the next decades. 


3.2. Ground robots 


The systematic design methods assist researchers in design choices, whereas the economic 
analysis considers allowable cost of a system. Only a few authors report design processes 
based in requirement engineering. The decisions about the hardware have influence on the 
robot performance, the complexity of algorithms and, eventually, the system costs. Some 
approaches for ground robots are reviewed with special interest in the platform, manipulator 
and end-effector implemented in several agricultural applications. 


Regarding to the platform, custom mobile robots are the most common choice [77]. The 
robustness of crawler and caterpillar platforms [78, 79] is described by several authors, but the 
wheeled robots [80] are also shown as good choices due to their simplicity. All these platforms 
can integrate approaches such as GPS [43], odometry [50], line guidance [81], path plans [82] 
and manned approaches [83] as navigation strategy. Other robots take advantage of irrigation 
pipes or rails in the field [84, 85] to achieve their displacement. 


The most common configuration of the manipulators is near 3 DoF (Degrees of Freedom) 
in a custom-made development [86, 87]. This situation can be associated to the high cost of 
implementations involving commercial manipulators [39]. Although the manipulators are 
commonly custom made, neither analysis nor explanation for the number of DOF selected is 
usually detailed by the authors. The manipulator movements are just simplified, and the high 
numbers of DOF are avoided. 


Finally, most end-effectors are designed to operate with two fingers [44, 79], since most of the 
grasps can be performed by them, and they are the smallest suitable mechanical architectures 
for grasping hand devices. In addition, grasping is commonly achieved by suction grippers 
[88]; for that reason, the end effectors are mainly actuated by electrical and pneumatic systems. 
A great number of the end effectors are custom made [89, 90]; this design preference can be 
associated to the huge diversity of tasks to be performed, as well as the several kinds of fruits 
and vegetables to be handled. Additionally, it is a common practice trying to solve the prob- 
lems originated by the mobile platform and the manipulator through the end-effector behav- 
iour; this situation leads to greater customization of the grippers. 


For comprehensive reviews of the agricultural robotics literature and further exemplification 
of their ground robot applications, we refer the reader to the readings [91, 92]. 
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3.3. Special robots 


The traditional ground robots present several limitations related to the constraints of wheeled 
and caterpillar motion. Additionally, the use of aerial robots is not always possible, especially 
when the task should be performed on surface or indoor. This section reports some cases in the 
literature where robots with alternative locomotion systems are applied to agricultural tasks. 


Sphere robots are systems whose movements are produced by instability. This type of robots 
is used in several applications and scenarios, such as exploration [93] and surveillance [94]. 
ROSPHERE is described by their authors as a new low-cost spherical robot for measuring soil 
temperature and moisture in precision agriculture [95]. In comparison to wheeled robots with 
similar size and capabilities, the ROSPHERE is much lighter and robust in irregular terrains. 


Several robots are designed by bioinspiration, which tries to replicate the biological evolution. 
For instance, the engineers have noticed that hexapod insects are able to walk in all terrains, 
so they are replicating their physiognomies and walking patterns. This is the case of Prospero, 
which is a prototype of hexapod robot that can plant, tend and harvest autonomously [96]. 
Another example are the RoboBees of Harvard University, which are micro-aerial robots 
inspired in bees [97]. These robots are being applied for distributed environmental monitor- 
ing and assistance to crop pollination. 


3.4. Multi-robot systems 


Sometimes, single robots are not able to perform some complex tasks (e.g. those that require 
coordinated actions in multiple locations) or to perform simple tasks in the required time (e.g, 
when the tasks must be performed in large areas). In these cases, robot teams might provide some 
advantages over single robots, such as their effectiveness, efficiency, flexibility and fault tolerance. 


Most of the cases of robot teams for agricultural tasks reported by the literature are homo- 
geneous. Using a fleet of UAVs instead of a single UAV for collecting data in large areas is 
common, and there are multiple techniques for area distribution and path planning [98]. For 
instance, a team of small UAVs with low-cost cameras can be applied to control the exploita- 
tion and management of water [99], obtaining the same results than a single UAV equipped 
with a better camera and providing the operation with more robustness. 


Nevertheless, some agricultural applications require heterogeneous robot teams. The most 
common situation is when the task consists of operations that are more effective from the 
air and others that are more effective from the ground. For instance, aerial robots are able to 
efficiently cover large fields taking pictures and collecting data (e.g. distribution of water or 
location of weeds), whereas ground robots can actuate on the crops with more robustness and 
precision (e.g, watering or applying treatments) [9]. 


Therefore, the heterogeneous multi-robot systems often combine the advantages and com- 
pensate the drawbacks of different robots. The Section 4 describes with more details two 
different multi-robot systems applied to two different agricultural scenarios: outdoor agricul- 
ture and greenhouse farming. 
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4. Practical experience 


This section reports some of the experiences of the Robotics and Cybernetics Group (RobCib) 
in the context of robotics applied to agriculture. It is organized as follows: Section 4.1 sum- 
marizes the participation in the Robot Fleets for Highly Effective Agricultural and Forestry 
Management (RHEA) Project, whereas Section 4.2 describes the use of multiple robots for the 
environmental monitoring of greenhouses. 


4.1. RHEA project 


A good example of precision agriculture is the RHEA (Robot Fleets for Highly Effective 
Agricultural and Forestry Management) Project. It was carried out under the seventh frame- 
work program of the European Commission and identified as NMP-CP-IP 245986-2. RHEA 
activities finished on 31 July 2014. The project was focused on designing, developing and test- 
ing of a new generation of robotic systems for both chemical and physical (mechanical and 
thermal) effective weed management in the context of agriculture and forestry. 


The use of pesticides in agriculture helps to improve yields and to prevent crop losses. 
Nevertheless, pesticides include active ingredients that have adverse impacts on the envi- 
ronment and habitats. According to the “Agriculture, forestry and fishery statistics” [100] 
online publication of Eurostat, the sale of pesticides in European Union member states 
amounted to close to 400,000 tonnes in 2014. Due to their potential toxicity, the application 
of pesticides is strictly controlled by EU legislation since 1991 and previously by national 
regulations. Although the return of pesticides (the crops saved from pests and diseases) 
is approximately four times the investment [101], the indirect costs (the impact on human 
health and environment) are estimated to total approximately $10,000 million per year in the 
United States [101]. 


The farmers usually apply these pesticides by using traditional sprayers that distribute them 
uniformly over the complete field. Therefore, the aim of RHEA Project was to support the 
farmers by reducing the amount of pesticides applied without reducing the effectiveness of 
the treatments. This objective was reached by applying the pesticides with high precision only 
where it is required. This solution not only prevents the pernicious effects of the pesticides but 
also reduces drastically the economic cost of treatment. 


RHEA scope covers a large variety of European products, such as agriculture wide row crops 
(processing tomato, maize among others), close row crops (winter wheat and winter barley) 
and forestry woody perennials (walnut trees, almond trees, olive groves and multi-purpose 
open woodland). 


The project is based on the cooperation among aerial and ground vehicles to perform preci- 
sion agriculture tasks, namely weeds removal and trees fumigation. A complete description 
of the system as well as the result of the project can be found at Ref. [9]. 


Ground units are based on small Case New Holland Industrial tractors with some modifications. 
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In order to provide the system with the required autonomy, several kinds of sensors and 
actuators were deployed into the systems. 


A high precision Global Navigation Satellite System (GNSS) for autonomous outdoor navigation 
was used to allow the control system to accurately steer the robots to work on wide-row crops (with 
0.75 m-spaced rows). Additionally, a ground perception system was used to discriminate weeds 
from crops while travelling along crop rows, as well as a real-time tree canopy detection system. 


The tractors were also endowed with three kinds of actuators, namely a patch sprayer aim- 
ing at reducing herbicide use by approximately 75%, a canopy sprayer to reduce the use of 
pesticide in canopy spraying by approximately 50% and a mechanical/thermal tool to destroy 
90% of the detected weeds. 


Additionally, the tractors were provided with a communication equipment, a sustainable 
energy system and a safety systems for humans and animals detection. Figure 1 shows the 
ground units with the mentioned actuators. 


In order to plan the activities of the ground units, the RHEA concept includes the support 
of aerial imagery by using drones. Thus, the system used a fleet of last-generation hexacop- 
ters provided by Air Robot that rely on high payload and extraordinary stability (shown in 
Figure 2). These features allow taking steady pictures with high-quality cameras in large open 
fields. As result, high-resolution images of the open field are obtained, in order to provide the 
ground units with the locations of weeds to remove them. 


Usually, weeds are not uniformly spread over the fields but are located in patches. For this 
reason, the first step of the mission is to obtain high-precision aerial images to locate the 
patches in the field. The location of the patches will allow defining optimal paths for the trac- 
tors. This task requires a thorough planning of the flights, which was our main task in this 
project. Several area coverage planning techniques were applied taking into account spatial 
and temporal requirements, in order to generate optimal and safe flight plans for the drones. 
A complete description of the aerial mission can be found in Refs. [102, 103]. 


Figure 1. Ground units of RHEA project. 
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Figure 2. Aerial units of RHEA project. 


The drones were equipped with two high-resolution cameras (4704 x 3136) mounted on a 
gimbal system. A multi-spectral device that includes visible and near infrared (NIR) channels 
was selected to maximize the robustness under different light conditions. Moreover, in order 
to preserve the complete colour information, the solution uses a coupling of two commercial 
still cameras: one of them modified to provide the NIR channel. The flights were performed 
with an elevation of 60 m to obtain a resolution of 1 cm per pixel with each image covering 
approximately 40 x 30 m and overlapping 6076 with the consecutive ones. 


RHEA provides the farmer with a ground station with a graphical user interface (GUI). This 
interface allows the farmer to create and launch the mission. Thus, the operator has to define 
the area by entering the points that limit the field. 


Once the limits of the field have been established, the system creates a flight plan for each 
robot of the fleet. After testing several optimization techniques, back and forth motion move- 
ments are applied for planning the fleet trajectories, in order to increase the situational aware- 
ness of the operator. Although the drones are able to autonomously perform take-off and 
landing operations, the pilot needs to provide altitude commands to ensure a safe operation. 


Later, a mosaicking procedure is developed: colour and NIR images from the two cameras 
are joined in a unique four-channel picture. An approach based on the Fourier-Mellin (FM) 
transform was successfully developed and tested. This approach identifies rotation, transla- 
tion and scale changes between images by means of Fourier spectrum analysis. To cope with 
large-sized camera images, which imply non-linear transformations, the original images are 
partitioned into a set of small image portions, where the FM identification process is executed 
iteratively. Then, a global homographic transformation model is computed including lens 
radial distortion. A registration accuracy of 0.3 pixels is obtained [17]. After obtaining a global 
image of the fields, the patch detection should be performed. 
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Detecting weeds when crop and weed plants are at early phenological stages is a challenge. 
The proposal of RHEA overcomes it by using high-spatial-resolution imagery and object- 
based image analysis (OBIA), taking into account the relative position of the weeds to the crop 
lines, so that every plant that is not located on the crop row is considered a weed [104]. As a 
result, a weed patch map is created using a grid of 0.5 m. The size of this grid is customizable 
according to the requirements of the herbicide-spraying machinery. 


Later, the Ground Mission Planner is executed by the operator in the base station. It deter- 
mines the configuration of the ground vehicles (number and type of vehicles), as well as the 
plan for each one to efficiently apply the treatment. Simulated annealing and basic genetic 
algorithms are used to find the optimal solution that minimizes either the task cost or the time 
required cost, whereas a non-dominated sorting genetic algorithm (NSGA-IT) is employed as 
a proper approach for simultaneously minimizing both criteria [105]. 


Once defined the optimal paths, the ground mission starts. During the mission, the ground 
perception system detects weeds, both inter and intra row. This system is based on a SVS- 
VISTEK camera connected to the high-level decision-making system computers for acquiring 
images and running the relevant vision algorithms [106]. The operation speed of Unmanned 
Ground Vehicles (UGVs) was fixed at 0.83 m/s, and the region of interest (ROI) for the ground 
perception system was defined to be 3-m wide and 2-m long located in front of the UGV. 
Weed detection relies on the spatial identification of crop rows. Thus, determination of crop- 
row positions with respect to the UGV becomes a key task for weed patch detection and UGV 
guiding. Weed detection system also generates the orders to the sprayers to activate precisely 
when nozzle passes over the weeds. 


4.2. Environmental monitoring in greenhouses 


As mentioned above, one of the agricultural applications where the robots can work is 
the environmental monitoring of greenhouses. This task cannot be performed by humans, 
because it requires a continuous work under the harsh conditions of greenhouses. The alter- 
natives of the robots in this task are the fixed sensors, which are not able to capture the spatial 
variability of the environmental conditions, and the sensor networks, which cannot be moved 
during the operation to the points of interest. 


The proposed solution is a multi-robot system that measures the environmental variables 
and collects their spatial and temporal variability. This information is the key to control the 
conditions of the crops, which determine both the productivity and the quality of greenhouse. 
As shown in Figure 3, the multi-robot system is split into small teams with ground and aerial 
units that work in specific areas. A base station controls the mission, coordinating the actions 
of the robots, as well as collecting and storing their measurements. 


The first work used a mini-UAV to measure air temperature, humidity, luminosity and car- 
bon dioxide concentration [5]. The sensors shown in Table 3 were selected to measure these 
variables regarding their size, weight, range, resolution and cost. These sensors were inte- 
grated by means of a Raspberry Pi computer, which collects the measurements and sends 
them to the base station via Wi-Fi network. 
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The main contributions of this work were the aerodynamics study of quadcopter and the real 
experiments with the sensory system. The results validated the use of quadcopter as sensory 
system and determined the optimal location of sensors: the centre and top of the quadrotor 
frame. A subsequent work developed a chamber to measure the concentration of gases mini- 
mizing the influence of propellers and obtaining less errors and fluctuations [107]. 


The second work introduced a medium-size UGV to measure ground temperature and 
humidity [6]. Specifically, we used the distance infrared temperature sensor and the contact 
conductivity humidity sensor collected by Table 3. Additionally, the work studied the path 
planning and following strategies to cover the greenhouse in the minimum time. 


The aerial and ground robots have different strengths and weaknesses. The aerial robots can 
move fast and agile through the corridors and reach any point in the 3D space. On the other 
hand, the ground robots have more autonomy to cover the greenhouse and more robustness 
to avoid accidents. Therefore, a heterogeneous team can take advantage of the potential of 
both types of robots [7]. 


Figure 3. Scheme of multi-robot system for environmental monitoring of greenhouses. 
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Variable Sensor Robot Controller 
Air temperature RHT03 UAV/UGV Raspberry Pi 
Ground temperature MLX90614 UGV Arduino 

Air humidity RHT03 UAV/UGV Raspberry Pi 
Ground humidity SEN92355P UGV Arduino 
Luminosity TSL2561 UAV/UGV Raspberry Pi 
CO, concentration MG811 UAV/UGV Raspberry Pi 


Table 3. Experience in environmental monitoring in greenhouses. 


The team strategy of the multi-robot system is the following: the UGV carries the UAV on 
a platform while it develops its tasks, and when it is required, the UAV takes-off, performs 
some tasks and lands on the UGV. In this manner, the multi-robot system can avoid the obsta- 
cles in the corridors, as well as find the source of anomalous measurements. 


The main future challenges of this research line are related to the navigation of the UAVs 
and the autonomy of UGV and UAV. The navigation of UAVs in the greenhouse is a chal- 
lenge because the scenario is closed and has high occupancy. The autonomy of both robots is 
required for the continuous operation and needs better batteries and charging systems. 


5. Conclusions 


In recent years, the robots have found their own place in agriculture. This chapter addresses 
the main fields of application (precision agriculture, greenhouse farming, and seeding and 
harvesting), analyzes the aerial, ground and special robots used in these applications, and 
describes two research projects related to precision agriculture and greenhouse farming. The 
main conclusions of these sections are summarized below. 


Precision agriculture seeks to apply multiple technologies to acquire knowledge about the 
spatial and temporal variability of crops. Among other technologies, the use of aerial robots to 
build maps of the fields and detect weeds or irrigation deficits, and the application of ground 
robots to apply accurate treatments to plants must be remarked. In addition, greenhouse 
agriculture has included robots for multiple tasks, such as the monitoring of environmental 
variables, which is important for the control of the conditions of crops, and the watering and 
spraying of plants. Finally, although the use of robots for seeding and harvesting is in an ear- 
lier step of development, a series of techniques of perception, positioning and grasping have 
been developed. 


The most widely used robots in agriculture are UAVs and UGVs. Aerial robots are usually 
applied to acquire information of the fields by taking advantage of the altitude. Although the 
first agricultural aerial robots were fixed-wing UAVs, nowadays the multi-rotors are more 
popular due to their flexibility. On the other hand, ground robots are usually used to act on 
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the crops. The most common configurations are wheeled and caterpillar robots, and some of 
the most relevant issues are the selected location and navigation algorithms. Nevertheless, 
other designs such as spherical or bio-inspired robots are gaining interest, as well as the use 
of multi-robot systems that can go further than single-robot ones. 


Finally, the chapter summarizes two different projects that address the application of robots 
in agriculture. The RHEA project uses aerial robots to locate weeds within the fields and 
ground robots to apply localized treatments on them. The other project introduces ground 
and aerial robots in greenhouses to take measurements of environmental variables. A series 
of lessons have been learned from these experiences, such as the potential of robots as mov- 
ing sensory and actuation systems, the difficulties of navigation in unstructured scenarios, 
the power of cooperation with heterogeneous fleets and the limitations imposed by the 
autonomy of robots. 
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